The present disclosure relates to an image registration device, an image registration method, and an image registration program for performing registration between a live view and an associated image associated with surgery of a surgical site.
In recent years, surgery simulation using three-dimensional medical images is becoming popular. The surgery simulation refers to simulation of an actual surgical procedure with visualizing a tissue or an organ, which is the objects of the surgery, and surrounding structures thereof from medical images. For example, with simulation of partial resection of the liver, tissues, such as the liver, the portal vein, veins, arteries, the body surface, bones, and a tumor, are extracted from a tomographic image, such as a CT (Computed Tomography) image or an MRI (Magnetic Resonance Imaging) image, and a three-dimensional image visualizing the extracted tissues is generated. This images is used to make a surgical plan by, for example, calculating, with a computer, an area to be resected of the liver region including a tumor in the liver, for example.
On the other hand, there is a demand for viewing such an image used for the surgery simulation (which will hereinafter be referred to as “simulation image”), as described above, as a reference during surgery. For this reason, such a simulation image may be printed on paper and brought in the surgery room, or may be displayed on a display device installed in the surgery room. Then, the surgeon conducts the surgery with looking at the actual surgical site of the patient, and the simulation image printed on paper or displayed on the display device for confirmation.
It is, however, very troublesome to conduct surgery with alternately looking at the patient and the simulation image, and there is a demand for displaying a simulation image superimposed on an image of the actual object of the surgery. To meet this demand, some approaches have been proposed for obtaining a live view formed by a plurality of images by imaging the surgical site during surgery, and displaying a simulation image superimposed on the live view. For example, Japanese Unexamined Patent Publication No. 2013-202313 (hereinafter, Patent Document 1) proposes an approach which involves attaching an optical sensor or a magnetic sensor to a camera for imaging the surgical site, to a surgical tool, or directly to an organ and performing calibration, i.e., initial registration, of the relative positional relationship among them, and moving a simulation image relative to change of the position and orientation of the camera and movement of the organ after the initial registration, to thereby display the simulation image that is registered with and superimposed on the view of the surgical site.
Also, an approach which involves placing a marker on a surgical site, detecting the position of the marker with a sensor to register a simulation image with a live view of the surgical site, and displaying the simulation image on a head-mount display (see Japanese Unexamined Patent Publication No. 2010-259497 (hereinafter, Patent Document 2), and an approach which involves embedding a marker in a surgical site, detecting the marker with a sensor to register a simulation image with the surgical site, and displaying the simulation image registered with the surgical site on a monitor (see Japanese Unexamined Patent Publication No. 2010-200894 (hereinafter, Patent Document 3) have been proposed.
On the other hand, approaches where registration between a live view of a patient obtained by imaging and a simulation image is performed without using a sensor, or the like, have been proposed. For example, PCT Japanese Publication No. 2015-523102 (hereinafter, Patent Document 4) proposes an approach which involves generating a model of an organ from a three-dimensional image, and calculating an affine transformation function between the model of the organ and each frame of an ultrasonic image to display the model of the organ superimposed on the ultrasonic image real-time, and superimposing the model of the organ on the ultrasonic image by transforming the model of the organ based on the affine transformation function. Further, PCT Japanese Publication No. 2010-532035 (hereinafter, Patent Document 5) proposes an approach for superimposing a simulation image on a stereoscopic live view, where the simulation image is superimposed on a taken image using the parallax of the taken live view.
Further, in order to achieve registration between the taken live view and the simulation image, it is necessary to perform registration between time series images forming the taken live view. In order to achieve accurate registration, it is necessary to determine appropriate feature points used in the registration. For this purpose, an approach is proposed, which involves detecting feature points in two images obtained in time series included in the live view, and detecting movement of the imaging device by calculating movement of each of feature points that are selected based on the directions of loci, amounts of loci, distribution, etc., of the feature points (see Japanese Unexamined Patent Publication No. 2010-14450 (hereinafter, Patent Document 6)). Also, an approach is proposed, which involves setting a region of interest on first one of two images obtained in time series included in the live view, determining an area on the obtained images from which feature points usable for registration of the region of interest can be extracted, and performing registration between the two images using the feature points determined within the area (see Japanese Unexamined Patent Publication No. 2007-257287 (hereinafter, Patent Document 7)). The approaches taught in Patent Documents 6 and 7 allow selecting appropriate feature points used in registration between two images, thereby allowing accurate registration between two images.